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Abstract 

Cloud  detection  is  a  crucial  step  in  any  climate  modelling  or  prediction.  Multi-angle  Imaging 
SpectroRadiometer  (MISR)  was  launched  in  1999  by  NASA  to  provide  9  angle  and  4  band  data 
to  retrieve  or  estimate  the  cloud  height  and  hence  cloud  detection.  However,  cloud  detection 
even  with  MISR  data  has  been  proven  very  difficult  over  ice  and  snow.  In  this  paper,  we  bypass 
the  cloud  height  estimation  step  to  directly  tackle  cloud  detection  by  using  features  of  ice  and 
snow  (no  cloud)  pixels  from  different  MISR  angles.  We  propose  the  linear  correlation  matching 
classification  (LCMC)  algorithm  based  on  Fisher  linear  correlation  tests.  We  compare  LCMC 
with  the  Steroscopically-Derived  Cloud  Mask  (SDCM),  which  is  the  cloud  mask  from  MISR  Level 
2  Top-of-the  atmosphere  Cloud  algorithm  (known  as  L2TC),  and  find  that  LCMC  gives  more 
coverage  and  more  robust  results  judged  by  visual  inspection  of  finer  resolution  images.  LCMC 
can  also  detect  the  very  thin  clouds  most  of  the  time.  Moreover,  LCMC  is  computationally 
much  faster  than  L2TC  and  easier  to  implement.  We  hope  to  combine  LCMC  with  L2TC  in 
the  future  to  improve  the  accuracy  of  the  L2TC  cloud  height  retrieval. 


Index  Terms  —  Cloud  detection,  ice  and  snow,  Multi-angle  Imaging  SpectroRadiometer  (MISR), 
linear  correlation. 


1  Introduction 


MISR  was  built  for  NASA  by  the  Jet  Propulsion  Laboratory  (JPL)  in  Pasadena,  California,  and 
is  one  of  five  instruments  launched  into  polar  orbit  aboard  NASA’s  Terra  spacecraft  in  December 
1999.  MISR  is  part  of  NASA’s  Earth  Observing  System  (EOS)  program,  which  was  conceived  to 
provide  data  for  in-depth  scientific  investigations  of  Earth’s  processes  as  a  unified  system.  These 
data  facilitate  studies  of  critical  processes  such  as  global  warming  and  greenhouse  effects. 

Clouds  play  a  major  role  in  Earth’s  climate  by  differentially  warming  and  cooling  the  Earth.  It  is 
widely  acknowledged  that  clouds  introduce  uncertainties  in  radiative  feedback  to  the  climate  system, 
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Figure  1:  MISR  data  registration.  Only  three  of  the  nine  angles  are  shown  here  (An  (nadir),  Bf 
(45.6°  forward)  and  Df  (70.5°  forward)). 


and  that  this  poses  a  most  formidable  obstacle  to  climate  prediction  and  weather  forecasting.  In 
particular,  in  polar  regions  where  scenes  are  dominated  by  ice  and  snow,  the  net  radiative  impact 
is  hard  to  discern  (Diner  et  al,  1999a;  Charlock  and  Ramanathan,  1985;  Li  and  Leighton,  1991). 
Some  studies  indicate  that  clouds  tend  to  warm  the  surface-atmosphere  system  over  highly  reflective 
surfaces  (Stephens  et  al,  1981;  Nakamura  and  Oort,  1988),  but  others  studies  suggest  clouds  have 
a  net  cooling  effect  (Li  and  Leighton,  1991;  Schweiger  and  Key,  1994).  Moreover,  most  aerosols 
are  bright  particles  that  reflect  sunlight  back  to  space.  They  tend  to  cool  the  surface  by  reducing 
the  amount  of  solar  radiation  absorbed  at  the  surface.  Aerosol  information  cannot  be  retrieved  in 
the  presence  of  clouds,  so  good  cloud  identification  methods  are  essential  for  aerosol  studies.  The 
ability  to  detect  clouds  in  the  polar  regions  is  therefore  critical  to  understanding  Earth’s  radiation 
budget,  and  the  role  of  clouds  in  the  climate  system. 

The  MISR  instrument  offers  a  new  and  promising  avenue  for  cloud  detection  methods.  MISR 
has  nine  cameras  looking  at  the  Earth  from  different  angles  simultaneously.  The  nine  camera  angles 
are  70.5°,  60°,  45.6°,  26.1°  forward,  nadir  and  26.1°,  45.6°,  60°,  70.5°  afterward  (referred  to  as  Df, 
Cf,  Bf,  Af,  An,  Aa,  Ba,  Ca,  Da,  respectively).  Each  angle  has  four  observing  bands  (red,  green, 
blue,  and  near- infrared) .  The  spatial  resolution  is  275m  by  275m  in  red  band  and  in  all  bands 
of  the  An  camera,  and  1.1  km  by  1.1  km  in  green,  blue,  and  near-infrared  bands  of  the  off-nadir 
cameras.  The  instrument  about  produces  six  million  measurements  per  orbit  with  16  orbits  per  day 
(Muller  et  al,  2002).  This  multi-angle  information  permits  direct  retrieval  of  cloud  heights,  which 
is  impossible  using  single  angle  views  (Diner  et  al,  1999a).  MISR  registers  each  pixel  to  the  Earth’s 
surface  ellipsoid.  The  higher  an  object  is,  the  greater  the  disparity  between  its  ellipsoid-projected 
positions  as  seen  from  different  angles.  Fig.  1  shows  the  registration  for  three  of  nine  MISR  cameras 
(Df,  Bf  and  An).  Simple  trigonometry  allows  one  to  determine  cloud  height,  and  height  can  then 
be  thresholded  to  determine  whether  a  cloud  is  present  or  not.  This  is  the  basis  of  the  the  MISR 
cloud  height  retrieval  algorithm  (L2TC)  and  the  Steroscopically-Derived  Cloud  Mask  (SDCM).  It 
works  well  in  many  cases,  but  fails  to  classify  a  substantial  portion  of  pixels  correctly  when  applied 
over  ice  and  snow  scenes  because  it  is  difficult  to  identify  matching  pixels  in  different  views  against 
the  bright  background. 

Fig.  2  displays  an  RGB  image  of  the  An  view  over  the  East  Siberian  Sea,  Russia,  obtained  on 
July  24,  2002  (MISR  orbit  12986,  blocks  22-24).  The  left  panel  is  at  1.1km  by  1.1km  resolution, 
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while  the  right  panel  is  at  275m  by  275m  resolution  for  the  middle  sized,  boxed  region  in  the  left 
panel.  It  is  almost  impossible  to  identify  clouds  in  the  l.lkrn  by  1.1km  image  by  visual  inspection. 
The  algorithms  do  find  clouds  which  can  be  confirmed  looking  at  the  finer  resolution  in  the  right 
panel. 

Recall  that  when  a  cloud  is  present,  it  is  registered  at  different  locations  for  images  taken  at 
different  angles  (cf.  Fig.  1).  The  challenge  of  retrieving  cloud  height  is  thus  that  of  matching  the 
same  cloud  scene  in  images  representing  different  angles.  After  matching  at  three  non-symmetric 
angles,  we  can  use  angle,  flight  time,  and  geometry  to  solve  for  cloud  motion  and  height.  Matching 
over  ice  and  snow  is  hard  since  the  cloud  textures  are  easily  confused  with  those  of  ice  and  snow. 
Our  LCMC  method  approaches  the  problem  from  another  point  of  view.  It  is  based  on  matching 
the  ice  and  snow  pixels  when  no  clouds  are  present.  This  provides  better  speed  and  accuracy. 

In  section  2,  we  review  briefly  a  few  stereo  matchers  used  in  MISR’s  algorithms.  We  describe 
our  LCMC  algorithm  in  section  3.  The  comparision  of  LCMC  with  the  MISR  algorithms  and  an 
example  of  LCMC  thin  cloud  detection  are  given  in  section  4.  Section  5  concludes  with  a  discussion 
and  ideas  for  future  work. 


2  Review  of  Existing  Algorithms 


Here  we  review  two  algorithms  that  have  been  used  or  investigated  by  the  MISR  science  team.  The 
algorithm  in  use  now  is  called  MISR  level  2  top-of-the  atmosphere  cloud  algorithm  (L2TC)(c.f. 
Dinner  et  al,  1999).  Pyramidal  Gruen-Otto-Chau  adaptive  least  squares  correlation  matcher  (P- 
Gotcha  matcher)  is  another  matcher  that  had  been  considered  for  MISR  cloud  height  retrieval  in 
JPL  (c.f.  Gruen,  1985;  Otto  and  Chau,  1989;  Holden  et  al,  1993). 


2.1  L2TC  Cloud  Height  Retrieval  Algorithm 

L2TC  uses  Df-Bf-An-Ba-Da  cameras  for  cloud  top  motion  retrieval  (estimation)  and  uses  Af-An-Aa 
cameras  for  cloud  height  retrieval  (estimation).  First  the  top  cloud  motion  is  estimated  or  retrieved 
and  it  is  used  to  estimate  or  retrieve  the  cloud  height.  Thresholding  the  cloud  height  gives  the 
cloud  mask,  Steroscopically-Derived  Cloud  Mask  (SDCM). 

Raw  MISR  measurements  are  transformed  into  Bi-directional  Reflectance  Factors  (BRF’s),  i.e. 
the  ratio  of  observed  radiance  at  a  particular  illumination  and  view  geometry  and  the  radiance 
that  would  be  obtained  under  the  same  conditions  with  a  target  consisting  of  a  perfect  lambertian 
reflector  (cf.  Diner  et  al,  1999b).  The  275m  resolution  red  band  is  used,  because  it  provides 
high  resolution  data.  However,  the  classification  label  is  given  only  at  l.lkrn  resolution  to  reduce 
computation.  (In  this  paper,  we  concentrate  mainly  on  the  red  band  for  the  same  reason,  but  would 
like  to  investigate  the  use  of  other  bands  for  further  improvements  in  the  future.) 

L2TC  estimates  cloud  motion  and  height  in  two  separate  stages,  although  they  can  be  solved 
with  two  linear  equations  simultaneously  after  a  match  is  found.  This  choice  reflects  a  compromise 
between  computational  speed  and  classification  accuracy.  At  the  first  stage  L2TC  retrieves  the 
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cloud-top  motion  over  non-overlapping  blocks  of  size  70.4km  x  70. 4km  for  computational  speed. 
It  is  not  feasible  to  match  every  pixel  in  three  angles  at  high  resolution,  especially  for  the  most 
oblique  angle  (Df).  Cloud  movement  in  a  70. 4km  block  is  assumed  constant,  and  in  particular 
cloud  vertical  movement  is  assumed  negligible  during  the  7-minute  observation  period.  L2TC  uses 
Nested  Maxima  Stereo-Matcher  (NM)  to  find  matching  blocks  using  three  cameras  (An-Bf-Df). 
NM  is  a  feature-based  matcher  designed  to  run  through  large  amounts  of  data  at  high  speed.  A 
nested  maximum  is  defined  as  a  central  pixel  within  a  5  pixel  vertical  column  such  that  the  BRF 
decrease  monotonically  in  value  for  two  pixels  on  both  sides  of  the  maximum  pixel.  L2TC  first 
searches  for  all  nested  maxima  in  both  the  reference  (Bf)  angle  image  and  the  image  to  be  searched 
in  (An  and  Df);  then  it  takes  one  maximum  from  the  reference  image  and  one  maximum  from  the 
searched  image  to  test  if  they  match  each  other.  For  an  11  by  21  pixel  patch  (window)  around  each 
target  pixel  (maximum  here),  M2  and  M3  (defined  below)  match  metrics  are  calculated  (Diner  et 
al,  1999b)  to  decide  on  a  match  (empirically  chosen  thresholds  1  for  M2  and  0.75  for  M3).  If  there 
is  no  match,  then  a  label  “no  retrieval”  is  given  for  the  block  and  hence  no  pixels  in  the  block  will 
have  cloud  motion  retrievals,  or  cloud  height  retrievals.  Also,  if  there  are  too  many  matches  in  the 
block,  cloud  motion  cannot  be  retrieved,  and  “no  retrieval”  is  also  given  to  such  blocks. 


The  M2  metric  is  defined  as: 


Sm2  = 


|  r  A' -  !i  ; )  R 
''-Rmax  Rmin 


r  C (xi  ,yj )  C 

tCm  ax  Cmin  ■ 


CM  2 


with 


CM2 


|| -R(xj,yj)  R 

■  ■  Rmax  Rmin 


(1) 

(2) 


where  R(xj,yj)  is  the  reference  pixel  BRF  at  C(xi,yj)  is  the  corresponding  value  in  the 

comparison  image;  Rmax ,  Rmin  are  the  maximum  and  minimum  values  within  the  reference  patch, 
respectively;  Cmax ,  Cmin  are  the  maximum  and  minimum  values  within  the  comparison  patch, 
respectively;  R  is  the  average  value  within  the  reference  patch;  C  is  average  value  within  the 
comparison  patch.  M3  uses  medians  rather  than  means  to  define  a  metric  in  a  similar  way  (cf. 
Diner  et  al,  1999b). 


After  a  match  is  found,  L2TC  uses  the  following  equations  to  determine  cloud  motion,  even 
though  cloud  height  is  obtained  at  the  same  time  by  solving  these  equations: 

vc(tAn  ~  tBf)  ~  h(tan{0Bf)  -  tan(0An ))  =  (xAn  -  xBf)  (3) 

vc{tDf  -tBf)  -  h(tan(dBf)  -tan(dDf))  =  (xDf  -  xBf)  (4) 

where  vc  is  the  along-track  cloud  movement  speed  (cloud  motion);  h  is  the  cloud-top  height;  t  is 
the  time  when  the  given  camera  looks  at  that  location;  9  is  the  angle  of  view  direction  and  the 
zenith;  xAn  —  xBf  and  xBf  —  xBf  are  the  along-track  location  shifts  of  the  best  matched  patch  in 
An  vs.  Bf  and  Df  vs.  Bf. 


At  the  second  stage  of  L2TC,  cloud-top  motion,  vc,  for  the  block  is  assumed  known,  and  the 
An  and  Af  images  are  used  to  obtain  cloud  height  on  a  pixel-by-pixel  basis.  L2TC  finds  the  match 
in  An/Af  angles  and  solves  just  one  linear  equation  (3)  to  get  cloud  height,  if  we  replace  Bf  by  Af 
and  use  the  cloud  movement  speed  vc  got  from  the  first  stage. 
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M2  and  M3  are  also  used  to  retrieve  the  cloud  height  pixel  by  pixel.  L2TC  calculates  the  M2 
metric  of  a  small  pixel  patch  (e.g.  6  by  10)  extracted  around  the  target  pixel  chosen  for  matching, 
and  a  patch  around  each  comparison  pixel  in  the  search  window.  The  pair  of  patches  with  the 
lowest  metric  value  is  chosen.  The  match  must  pass  both  the  threshold  test  (e.g.  M2  metric  less 
than  threshold)  and  ambiguity  tests  (e.g.  metrics  at  other  locations  in  the  search  window  are  larger 
than  110%  of  the  metric  at  the  best  match)  to  be  accepted.  For  a  successful  M2  match,  M3  is 
applied  to  this  best  matching  location  to  determine  whether  M3  verifies  the  point  as  valid  match.  If 
a  match  can  not  be  found,  cloud  height  for  that  pixel  is  labelled  as  “no  retrieval”.  (The  thresholds 
for  M2  and  M3  are  derived  empirically  by  studying  match  quality  versus  metric  value  for  a  number 
of  different  scenes.  The  relationship  does  not  vary  with  scene  type,  and  are  specific  to  all  MISR 
scenes,  irrespective  of  the  types  of  clouds  present  in  the  scene.)  L2TC  detects  cloud  by  thresholding 
the  difference  of  cloud  height  estimation  and  terrain  height,  so  there  is  no  cloud  detection  result 
for  the  cloud  height  “no  retrieval”  pixel. 

The  main  drawback  of  L2TC  over  is  that  cloud  height  retrieval  fails  for  an  entire  block  when 
the  NM  matcher  fails.  This  happens  particularly  over  ice  and  snow,  because  the  local  maxima  of 
the  BRF’s  from  cloud  and  ice/snow  are  mixed  up.  When  the  cloud  motion  for  a  block  are  retrieved 
successfully,  M2  and  M3  sometimes  also  fail  to  find  matches  in  the  second  step.  The  right  panel  of 
Fig.  3  shows  the  “no  retrieval”  pixels  from  L2TC,  in  which  cloud  motion  is  retrieved  successfully  for 
all  the  blocks.  It  fails  on  18%  of  pixels.  The  blockiness  (70.4  x  70.4  km2)  in  the  motion  retrievals 
leads  to  blockiness  in  the  height  retrievals  as  well. 


2.2  P-Gotcha  Matcher 

The  P-Gotcha  matcher  is  based  on  a  parametric  regression  model,  refitted  at  candidate  matching 
pixels  sequentially.  It  assumes  smooth  reflectance  and  terrain  surfaces.  Hence,  a  linear  Taylor 
approximation  can  be  made  locally  and  gives  a  local,  linear  regression  model.  The  fitted  parameters 
give  the  location  coordinates  of  the  next  candidate  matching  pixel  relative  to  the  location  of  the 
current  candidate.  The  process  stops  when  parameters  fall  below  a  threshold,  or  the  candidate 
matching  pixel  becomes  stable. 

P-Gotcha  was  originally  selected  as  the  stereo-matcher  for  MISR  because  it  provides  sub-pixel 
acuity  by  interpolation  in  the  parametric  model  (typically  to  0.3  pixels),  and  can  provide  a  precision 
matrix  for  estimating  accuracy.  However,  P-Gotcha  is  exceedingly  slow  since  it  performs  least- 
squares  fitting  at  every  candidate  matching  pixel  (about  105  floating-point  operations  per  match, 
Muller  et  al,  2002).  Empirically  it  was  found  that  searches  started  more  than  3  pixels  away  from 
the  best  match  do  not  converge,  so  the  coverage  is  also  promising.  The  MISR  team  decided  not 
to  proceed  with  this  algorithm  as  the  operational  matcher  even  using  a  parallel  implementation. 
However,  when  it  does  converge,  P-Gotcha  provides  a  benchmark  by  which  to  judge  the  performance 
of  other  algorithms. 

In  summary,  over  ice  and  snow  neither  L2TC  nor  P-Gotcha  provide  good  coverage.  Cloud  de¬ 
tection  in  polar  regions  remains  an  impediment  to  making  full  use  of  MISR  multi-angle  capabilities 
in  this  area. 
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3  Linear  Correlation  Matching  Classification  (LCMC) 


We  propose  a  new  algorithm,  the  Linear  Correlation  Matching  classification  (LCMC)  algorithm. 
This  algorithm  can  be  used  for  classification  of  clear  pixels  over  ice  (hence  also  cloud  pixels  by 
exclusion) . 

3.1  Rational  Behind  LCMC 

The  rationale  behind  LCMC  is  as  follows.  Since  the  radiances  into  different  cameras  (angles)  at  a 
particular  band  from  the  same  scenery  are  approximately  proportional  to  the  intensity  of  the  light 
source  (sun),  they  should  be  strongly  correlated.  Over  ice  and  snow,  the  radiances  are  relatively 
high  and  this  correlation  is  obvious.  (In  contrast,  the  radiances  of  clear  sea  pixels  are  very  low,  so 
the  correlation  is  masked  by  random  noise.) 

Preliminary  data  analysis  via  scatter  plots  on  the  MISR  data  confirms  the  strong  correlation. 
Examples  of  such  scatter  plots  are  displayed  in  Fig.  4  for  the  smallest  windows  in  the  left  image  of 
Fig.  2  with  the  l.lkrn  by  l.lkrn  resolution.  The  upper  left  plots  correspond  to  the  upper  smallest 
window  in  Fig.  2  and  show  a  strong  linear  correlation  of  radiances  between  Bf  and  Df  in  red  band. 
When  there  is  no  cloud  or  the  cloud  does  not  change  too  much  in  the  7  minutes  observation  period, 
the  best  match  corresponds  to  a  very  high  linear  correlation  over  a  small  search  area.  Next  we 
describe  our  LCMC  algorithm. 


3.2  LCMC  Algorithm 

We  use  the  earth  surface  ellipsoid  projected  data.  The  proposed  method  works  for  any  three  non- 
symmetric  angles.  To  be  concrete,  we  will  describe  it  using  the  An,  Bf  and  Df  angles.  This  angle 
combination  also  provides  the  best  cloud  height  separation  among  all  forward  angles  (Diner  et  al, 
1999b).  (An,  Ba  and  Da  angles  are  reserved  to  validate  the  result.) 

A  small  patch  (9  by  9  pixels)  is  chosen  centered  at  the  pixel  to  be  classified,  using  Bf  angle  as 
the  reference  angle,  An  and  DF  angle  as  comparison  angles.  If  there  is  no  cloud  over  that  location 
from  all  three  angle  we  used,  and  the  terrain  height  is  not  too  high  from  the  earth  surface  ellipsoid, 
the  linear  correlation  coefficient  r  between  radiances  in  the  An  patch  and  the  Bf  patch  at  same 
location  should  be  very  strong.  If  we  move  the  center  of  the  Bf  patch  one  pixel  forward  or  backward 
along  the  flying  direction,  the  strong  linear  correlation  will  be  destroyed.  This  observation  holds 
also  between  Bf  and  Df,  see  the  upper  two  panels  in  Fig.  4.  From  equation  (3)  and(4),  the  camera 
angles,  flying  time,  and  geometry  show  that  the  cloud  can  not  be  registered  at  the  same  location 
in  An,  Bf  and  Df  angles  at  the  same  time  (cf.  Diner  et  al,  1999b).  So  if  there  is  cloud  over  the 
window,  the  An/Bf  and  Bf/Df  linear  correlations  at  the  same  location  will  not  be  much  stronger 
than  one  pixel  forward  or  backward,  see  the  lower  two  panels  in  Fig.  4. 

We  build  the  LCMC  classification  rule  based  on  this  observation.  We  calculate  the  linear 
correlation  coefficients  of  the  Bf  (reference  angle)  patch  with  An  (comparison  angle)  patches  one 
pixel  forward  (rpf).  same  location  (r^J,  and  one  pixel  backward  (r^J  respectively.  We  test  if  the 
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same  location  correlation  coefficient  rBs  is  greater  than  other  two  significantly.  The  test  is  Fisher’s 
linear  correlation  test  (Fisher,  1925).  We  transform  all  three  r  to  z  =  |  log((l  +  r)/(  1  —  r)).  If  the 
data  vectors  are  i.i.d.  and  follow  jointly  a  Gaussian  distribution  with  correlation  p,  z  approximately 
follows  normal  distribution  with  mean  ^log((l  +  p)  /  (1  —  p)),  variance  l/(n  — 3)  (here  n  =  81).  The 
scatter  plots  show  that  this  joint  Gaussian  assumption  holds  approximately  for  our  data.  (We 
ignore  the  dependence  between  the  pixels  in  the  same  small  patch  as  a  first  approximation.) 

For  the  Bf  and  An  angles  testes,  we  use  the  5%  level  to  test  the  null  hypotheses  ps3  <  PBf 
and  pb3  <  PBb-  They  are  rejected  when  data  fall  into  rejection  regions,  i.e. ,  {zb3  —  zsf  > 
1 .645 2/(81  —  3)}  and  {zb3  —  zsb  >  1.645a/2/(81  —  3)}.  We  carry  out  the  same  testes  for  Bf 
and  Df  angles.  A  clear  (no  cloud)  pixel  is  declared  if  all  four  null  hypotheses  are  rejected,  otherwise 
cloudy. 


3.3  Computational  Order  of  LCMC 

The  computation  of  LCMC  is  much  simpler  than  L2TC  and  P-gotcha,  since  LCMC  only  calculates 
six  correlation  coefficients  to  classify  each  pixel.  It  is  linear  in  the  number  of  pixels  processed.  We 
need  not  go  through  a  search  window  to  find  the  best  match,  which  is  the  most  expensive  part  of 
the  other  algorithms.  Matching  the  ground,  instead  of  the  cloud,  provides  us  such  advantages.  By 
design  it  also  has  a  100%  coverage. 

3.4  Angle  and  Cut-off  Points  Selection 

Since  there  are  more  angles  to  be  used  for  this  testing,  in  the  future  we  plan  to  explore  approaches 
to  combine  the  tests  (e.g.  multiple  testing)  to  set  the  cut-off  values  for  cloud  detection.  The  cut-off 
value  might  vary  according  to  the  pair  of  angles  used  in  the  test  since  the  angles  near  An  provide 
more  reliable  information  than  those  away  from  An  because  they  incur  smaller  time  differences 
when  observing  the  same  scene.  In  addition,  we  would  also  like  to  explore  the  possibility  of  using 
the  other  three  bands  to  improve  cloud  detection  based  on  the  red  band. 

By  introducing  the  statistical  testing  idea  into  the  model,  we  want  to  investigate  the  confidence 
levels  associated  with  the  matching  result.  We  hope  to  find  a  way  to  access  the  confidence  levels 
of  the  classification  results  along  this  direction.  We  need  “ground  truth”  from  other  resources  to 
work  with. 


4  Experimental  Results  of  LCMC 


A  case  study  is  carried  out  to  compare  LCMC  with  SDCM.  The  East  Siberian  Sea  data  (An  angle 
RGB  image  in  Figure  2)  is  used.  Since  the  Df  angle  does  not  cover  the  area  outside  of  the  big 
window  in  the  image,  we  constrain  our  comparisons  inside  of  the  window.  The  second  example 
shows  the  ability  of  LCMC  to  detect  the  very  thin  cloud  over  ice  and  snow. 
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4.1  Comparisons  Between  LCMC  and  SDCM 


As  described  in  L2TC  review  section,  the  SDCM  classification  algorithm  takes  as  input  the  275m  by 
275m  resolution  data  (red  band)  and  outputs  classification  labels  at  the  1.1km  by  1.1km  resolution. 
Because  of  time  constraint,  we  applied  the  LCMC  classification  algorithm  only  to  the  l.lkrn  by 
1.1km  resolution  data  (red  band  as  well),  and  obtained  classification  labels  also  for  the  l.lkrn  by 
l.lkrn  resolution.  A  288  by  361  pixel  sub-image  in  Fig.  2  (indicated  by  the  largest  box  in  the  left 
panel)  was  chosen  so  that  L2TC  gave  cloud  motion  retrievals  for  all  pixels  to  simplify  the  matter. 
But  the  matching  in  the  height  retrieval  of  L2TC  failed  at  18%  of  the  pixels.  LCMC  classified  all 
pixels  by  design.  Table  1  contains  the  comparisons.  For  LCMC  no  cloud  or  clear  pixels,  LCMC 
and  SDCM  agree  86.5%  of  the  time,  disagree  0.65%  of  the  time,  and  12.8%  of  the  time,  L2TC  has 
no  retrieval. 


SDCM  No  Cloud 

SDCM  Cloudy 

SDCM  Not  Retrieved 

Total 

LCMC  No  Cloud 

27.86% 

0.21% 

4.12% 

32.2% 

LCMC  Cloudy 

35.22% 

18.4% 

14.18% 

67.8% 

Total 

63.08% 

18.61% 

18.31% 

100% 

Table  1:  Comparing  the  LCMC  result  with  SDCM. 


We  investigated  the  pixels  not  classified  by  L2TC  (18%)  in  the  small  boxed  windows  in  the  left 
image  of  Fig.  2  -  we  zoomed  in  to  the  275m  by  275m  resolution  to  get  the  right  image  of  Fig. 
2.  Cloud  and  ice  can  now  be  distinguished  by  eye  at  this  resolution,  but  not  very  easily.  The  left 
panel  of  Fig.  3  shows  the  LCMC  classification  result  of  the  data  in  Fig.  2  (“No  cloud”  pixels  are 
coded  as  white  points  in  the  picture,  and  “cloud”  pixels  are  left  in  the  true  color).  The  white  points 
in  the  right  panel  of  Fig.  3  are  the  pixels  Where  L2TC  failed.  For  the  L2TC  unclassified  pixels, 
LCMC  seems  to  give  reasonable  results.  Future  validation  has  to  be  made  based  on  other  sources. 

The  scatter  plots  of  radiances  of  the  9  by  9  windows  in  Bf/Df  angles  are  shown  in  Fig.  4.  The 
upper  two  plots  are  from  the  upper  smallest  window  in  Fig.  2,  which  is  classified  as  “no  cloud”  by 
LCMC.  The  linear  correlation  coefficient  drops  sharply  when  we  moved  the  Df  window  one  pixel 
forward.  The  lower  two  plots  are  from  the  lower  smallest  window  in  Fig. 2,  which  is  labelled  as 
“cloudy”  by  LCMC  classification.  The  linear  correlation  gets  stronger  when  we  move  Df  window 
one  pixel  forward.  So  the  evidence  is  strong  for  LCMC  to  classify  a  pixel  as  “no  cloud”. 

The  largest  disagreement  of  LCMC  and  L2TC  occurs  in  the  category  of  LCMC  cloudy  and 
L2TC  no  cloud.  This  is  due  to  the  fact  that  the  LCMC  classifies  one  pixel  as  “no  cloud”  when 
there  are  no  clouds  over  the  9  by  9  window  at  all  the  three  angles  (An,  Bf  and  Df),  that  is,  it  does 
not  classify  the  pixel  as  “no  cloud”  based  only  on  the  An/Af  pair.  However,  such  pixels  are  most 
likely  classified  as  “no  cloud”  by  L2TC,  because  it  uses  only  the  An/Af  pair  to  do  the  cloud  height 
retrieval.  So  it  is  possible  to  improve  accuracy  of  the  L2TC  by  using  other  angles,  but  at  the  price 
of  a  larger  searching  window.  The  accurate  detection  of  “no  cloud”  or  clear  pixels  by  LCMC  is 
important  in  its  own  right  because  the  MISR  aerosol  retrieval  algorithm  needs  to  know  where  the 
clear  or  no  cloud  pixels  are. 


4.2  Comparing  LCMC  with  Expert  Labelled  Data 


We  also  use  400  expert  labelled  pixels  to  test  both  LCMC  and  SDCM.  They  are  from  MISR  orbit 
12986,  block  22-24  (c.f.  Fig.  2).  The  400  pixels  are  labelled  by  visual  inspection  on  the  275m 
resolution  data.  “No  cloud”  pixels  are  chosen  from  the  top  middle  or  middle  left  area  of  the  image, 
while  “cloudy”  pixels  are  from  the  top  left  and  bottom  left  of  the  image.  200  clear  pixels  and  200 
cloudy  ones  are  include  in  the  test.  All  test  pixels  are  from  areas  where  it  is  easy  to  separate  “cloud” 
or  “no  cloud”.  The  test  results  are  given  in  Table  2. 


Mis  classification  Rate 

No  Cloud 

Cloudy 

Total 

LCMC 

47.5% 

0% 

23.75% 

SDCM 

42% 

28% 

35% 

Table  2:  Mis  classification  rates  of  LCMC  and  SDCM. 


We  notice  that  we  use  1.1km  by  l.lkrn  resolution  data  in  LCMC  algorithm,  but  SDCM  is 
derived  based  on  the  275m  by  275m  resolution.  Our  algorithm  still  provides  12%  improvement 
over  all.  The  results  show  again  that  the  LCMC  is  very  good  at  picking  up  high  clouds.  It 
makes  very  few  mistakes  when  there  are  really  clouds  above  a  certain  height.  Over  the  “no  cloud” 
area,  the  LCMC  gives  comparable  result  as  SDCM,  and  both  algorithms  have  relatively  high  mis 
classification  rates.  One  reason  for  this  mis  classification  is  that  we  use  the  ellipsoid  projected  data 
in  the  LCMC  algorithm.  The  high  terrain  surface  will  be  registered  in  different  locations,  so  the 
LCMC  algorithm  fails  to  pick  it  up  when  no  cloud  presents.  We  plan  to  test  LCMC  on  the  MISR 
terrain  projected  data  to  see  possible  improvement.  In  the  next  stage,  we  also  plan  to  test  the 
LCMC  on  the  275m  by  275m  resolution  data  and  hope  it  can  find  the  very  low  cloud  (below  1km) 
in  the  finer  resolution. 


4.3  Detecting  High  and  Thin  Cloud  over  Ice  and  Snow  by  LCMC 

So  far  all  the  stereo  matching  algorithms  assume  no  shape  change  of  the  clouds  during  the  ob¬ 
servation  interval.  Cloud  shape  change  will  make  the  problem  much  more  difficult  for  the  nested 
maximum  (NM),  M2  and  M3  matchers  to  do  their  jobs.  In  contrast,  the  shape  change  does  not 
present  anything  more  difficult  for  LCMC  because  it  relies  on  features  of  clear  pixels  which  remain 
the  same  with  or  without  the  cloud  shape  change.  So  LCMC  should  work  well  in  detecting  high 
and  thin  cloud  over  ice  and  snow. 

Fig.  5  displays  a  RGB  image  at  the  An  angle  of  MISR  orbit  2461  and  block  28-31  and  the 
LCMC  result.  The  left  panel  is  RGB  picture  with  the  l.lkrn  by  l.lkrn  resolution,  and  the  right 
panel  is  the  LCMC  result  (“No  cloud”  is  coded  as  bright  points).  The  left  upper  part  of  the  image 
is  covered  by  high  and  thin  cloud,  it  was  detected  by  the  LCMC  algorithm  very  well.  L2TC  is  not 
applied  to  this  data,  because  the  data  was  collected  before  L2TC  started  to  be  used.  (The  “no 
cloud”  pixels  over  sea  are  not  classified  by  LCMC,  but  by  thresholding  the  Infrared  radiances.) 
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5  Discussions  and  Future  Work 


In  summary,  LCMC  classifies  the  pixel  as  “no  cloud”  when  the  evidence  is  strong.  LCMC  has 
full  coverage  to  pick  out  the  “no  cloud”  pixels.  By  design,  LCMC  algorithm  relies  mostly  on  the 
surface  spatial  information  of  the  ice  and  snow  surface.  Other  MISR  bands  can  be  used  given 
there  is  enough  energy  in  the  bands  reflected  from  the  ground.  We  tested  LCMC  using  green  and 
blue  bands  on  the  MISR  data  shown  in  Fig.  2  and  Fig.  5,  and  both  bands  provide  very  close 
classification  results  in  our  tests. 

LCMC  does  not  work  over  ocean  because  the  linear  correlation  is  masked  by  random  noises, 
since  radiance  is  very  low.  In  this  case,  MISR’s  Level- 1  cloud  detection  algorithm  easily  detects 
clear  pixels  over  deep  ocean,  by  thresholding  the  near  infrared  radiance.  So  we  can  simply  threshold 
the  near  infrared  band  to  exclude  carrying  out  LCMC  for  that  pixel. 

LCMC  can  be  used  to  preprocess  data  for  L2TC  cloud  height  retrievals.  After  we  get  high 
confidence  “no  cloud”  pixels  from  LCMC,  we  could  run  L2TC  only  for  the  “cloud”  pixels.  In  the 
L2TC  cloud  motion  retrieval,  we  will  then  be  more  confident  that  we  are  matching  clouds,  not  ice 
and  snow  surface  features.  Accuracy  and  speed  are  both  improved  in  this  step.  For  the  L2TC 
cloud  height  retrieval,  we  do  not  need  to  search  matching  patches  for  the  “no  cloud”  pixels.  This 
reduces  the  computational  burden  and  improves  coverage. 

High  topography  poses  difficulties  for  LCMC  because  high  altitude  terrain  will  be  registered  at 
different  locations  for  different  view  angles.  We  need  to  incorporate  terrain  height  information  into 
the  LCMC  algorithm.  Since  terrain  height  is  provided  by  an  external  source,  and  MISR  provides 
the  terrain  projected  data,  we  are  going  to  test  the  LCMC  algorithm  on  terrain  projected  data  in 
the  next  stage. 

In  the  next  step,  we  want  to  implement  LCMC  on  the  275m  by  275m  resolution  data.  At  this 
finer  resolution  we  expect  LCMC  to  provide  better  results,  because  high  resolution  provides  more 
data  for  a  much  smaller  window.  Matching  tests  are  carried  out  even  more  locally.  When  a  pixel 
and  its  neighborhood  are  not  coved  by  clouds,  and  there  are  clouds  somewhere  in  the  9. 9km  by 
9. 9km  window,  the  275m  by  275m  resolution  LCMC  will  pick  this  out  while  the  l.lkrn  algorithm 
misses  it.  We  also  expect  the  finer  resolution  data  will  help  us  to  find  the  low  clouds.  We  would 
also  like  to  experiment  with  selecting  angles  for  use  in  the  finer  resolution  scenario. 

We  will  validate  the  LCMC  result  with  the  cloud  classifications  over  snow  and  ice  from  other 
sources  Reliable  data  from  other  source  could  also  be  used  to  improve  selection  of  angles,  bands, 
and  window  sizes.  One  source  of  validation  data  is  the  North  Slope  of  Alaska  Surface-Satellite 
Cloud  Comparison  Data  Site  [15].  It  provides  cloud  microphysics  data  from  a  surface  observation 
site  (latitude=71.17  longitude=203.22)  in  Barrow,  Alaska.  It  is  operated  by  the  Department  of 
Energy,  and  its  main  purpose  is  to  provide  validation  data  for  satellite  cloud  sensors,  particularly 
those  on  the  TERRA  satellite.  Clouds  observed  with  sensors  operating  at  this  ARM  North  Slope 
of  Alaska  CART  site  have  been  classified  based  on  subjective  (human)  examination  of  data  from 
radar,  microwave  and  infrared  radiometers,  and  radiosondes. 
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Summary 


As  cited  in  Diner  et  al  (2002),  the  20th  Century  writer  Zora  Neale  Hurston  wrote:  ’’There  is  no 
single  face  in  nature,  because  every  eye  that  looks  upon  it,  sees  it  from  its  own  angles.”  MISR  looks 
at  the  Earth  from  nine  different  angles  and  provides  us  new  ways  to  understand  the  changes  taking 
place  on  our  home  planet.  In  this  paper,  we  considered  the  problem  of  detecting  clouds  over  ice 
and  snow  from  a  different  point  of  view:  matching  the  ground  rather  than  the  clouds.  MISR’s  data 
collection  method  makes  it  possible  to  derive  a  fast  algorithm  for  distinguishing  clear  ice  and  snow 
pixels  from  cloud-covered  pixels.  The  algorithm  also  holds  potential  for  detecting  the  thin  cloud, 
even  when  the  cloud  shape  changes.  In  the  future,  we  hope  to  extend  the  method  to  work  over 
other  land  scenes,  and  combine  it  with  L2TC  for  further  improvements. 
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Figure  2:  RGB  true  color  image  of  the  East  Siberian  Sea,  Russia  (obtained  July  24,  2002).  MISR 
orbit  12986,  block  22-24,  An  angle.  Left  image:  RGB  in  1.1km  by  1.1km  resolution  image;  Right 
image:  RGB  275m  by  275m  resolution  image  in  the  middle  size  window  of  the  left  image. 


Figure  3:  MISR  Orbit  12986,  block  22-24.  The  left  panel  is  the  LCMC  result,  with  “no  cloud” 
pixels  coded  as  white  points.  The  right  panel  highlights  the  L2TC  failed  pixels,  coded  as  white 
points  (The  cloud  motions  are  retrieved  successfully  for  all  70.4km  by  70. 4km  blocks.) 
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Figure  4:  Scatter  plots  of  the  radiances  from  Fig.  2.  The  upper  two  plots  are  from  the  upper 
smallest  window  in  Fig.  2:  the  left  one  plot  is  from  BF  and  DF  angles  without  moving  the  DF 
angle  patch,  and  the  right  one  with  DF  angle  patch  moved  one  pixel  forward.  The  lower  two  plots 
are  from  the  lower  smallest  window  in  Fig.  2:  the  left  is  the  plot  of  radiances  in  the  BF  and  DF 
angles  without  moving  the  DF  angle  patch,  and  and  the  right  one  with  the  DF  angle  patch  moved 
one  pixel  forward. 


Figure  5:  MISR  Orbit  2461,  block  28-31,  AN  angle.  The  left  panel  is  the  RGB  true  color  picture, 
1.1km  by  1.1km  resolution.  The  right  panel  is  the  LCMC  cloud  detection  result,  with  “no  cloud” 
coded  as  white  points.  The  high  thin  cloud  is  detected  by  the  algorithm. 
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